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ABSTRACT 


Monitoring traffic in urban areas is an important task for intelligent transport applica- 
tions to alleviate the traffic problems like traffic jams and long trip times. The traffic 
flow in urban areas is more complicated than the traffic flow in highway, due to the 
slow movement of vehicles and crowded traffic flows in urban areas. In this paper, a 
vehicle detection and classification system at intersections is proposed. The system 
consists of three main phases: vehicle detection, vehicle tracking and vehicle classi- 
fication. In the vehicle detection, the background subtraction is utilized to detect the 
moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the 
removal shadow algorithm is developed to improve the detection phase and eliminate 
the undesired detected region (shadows). After the vehicle detection phase, the vehi- 
cles are tracked until they reach the classification line. Then the vehicle dimensions 
are utilized to classify the vehicles into three classes (cars, bikes, and trucks). In this 
system, there are three counters; one counter for each class. When the vehicle is clas- 
sified to a specific class, the class counter is incremented by one. The counting results 
can be used to estimate the traffic density at intersections, and adjust the timing of 
traffic light for the next light cycle. The system is applied to videos obtained by sta- 
tionary cameras. The results obtained demonstrate the robustness and accuracy of the 
proposed system. 
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1. INTRODUCTION 


Traffic problems are one of the most problems encountered by the residents of large cities. Therefore, 
the traffic management companies have paid great attention to solve these problems. The first step in traffic 
analysis is the collection of traffic information. Several techniques have been developed for traffic data collec- 
tion; many detectors (such as loop, radar, infrared and microwave) are utilized to do this task. These detectors 
help traffic flow management by providing information about the level of the traffic density on the roads. How- 
ever, they have many drawbacks that lead to reduce their use, as their installation requires pavements cuts; in 
addition, their detection zone is small. In recent years, the vision-based systems have been widely used in 
traffic management, due to their advantages compared to electronic sensors. The vision-based systems extract 
useful data by covering wide-area detection with ability in determining the shape of the detection zone. The 
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first phase to analyze the traffic parameters is the vehicle detection. Several methods have been developed for 
the vehicles detection, and these methods can be grouped into two main approaches: texture-based approach 
and motion-based approach. The first one utilizes vehicle features like edges, corners, colors and so on; this 
approach is good to detect the stopping vehicles. While the other one depends on the movement of vehicles, 
it is widely used in intelligent transportation systems. There are two main methods to detect the moving ob- 
jects: optical flow and background subtraction. The optical flow is accurate in the detection of object motion 
and gives more information about the motion like the velocity and the motion direction. However, it has high 
computational time and is not suitable for real time applications [1]. The background subtraction is the most 
common method used in literature for detecting moving vehicles. In this work, the background subtraction is 
utilized to separate the moving vehicles from the background model. However, the shadows of moving vehi- 
cles are also detected; thus they are considered as a part of vehicle dimensions, and this leads to misclassify 
the different vehicles. To cope with this issue, we proposed an algorithm to remove the shadow region based 
on the edges of detected regions. This algorithm outperforms the other shadow removal algorithms. When the 
vehicles are detected without shadows, they will be tracked until they reach the classification line. Then the 
classification is performed to category them into a number of predefined types. The classification methods can 
be grouped into two groups, one group classifies the vehicles by measuring the vehicle dimensions [2], while 
the other utilizes machine learning techniques [3]. 


Various systems have been developed to detect and classify the vehicles. A real time system for vehi- 
cles detection and classification is described in [4]. The first phase of their work is the vehicle detection, they 
used the background subtraction to perform this task by employing the adaptive background update method, 
then the vehicles were tracked by using the association graph between the consecutive frames, finally they used 
the vehicle dimensions to classify the vehicles into two categories, small vehicles (cars) and big vehicles(vans, 
trucks and buses). In [3], they developed a system for vehicle classification. The moving objects(vehicles)are 
separated from the static objects(background objects) by using GMM method, and then they used an approach 
based on two level of support vector machine (SVMs) to classify the detected vehicles. The second level was 
employed for solving the occlusion problem. The vehicles were classified into four classes (bus, cars, minibus 
and trucks). In [5], an image processing system is proposed to detect and classify the vehicles on rear view 
vehicle. This system utilized the temporal median filter to establish the background model, and then the scene 
frame was subtracted from the background model to detect the moving vehicles. The classification was per- 
formed by using SVM method after extracting vehicle features by using deep convolutional neural network. 
They classified the vehicles into two types, the passenger vehicles and other vehicle class. In [2], a real-time 
traffic surveillance system has been proposed to measure traffic flow by detecting and counting the vehicles. 
The background model was performed by using the temporal information of the mean and standard deviation 
of gray level distribution in consecutive frames for each point. After the segmentation process, each detected 
object was bounded by a rectangle box, and then the calculation of object features (height, width and aspect 
ratio) was established to achieve a robust and accurate classification. Then the vehicles were classified into two 
classes (cars and bikes). In this paper, a vehicle detection and classification system at intersections is proposed. 
The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the 
vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture 
of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection 
phase and eliminate the undesired detected region (shadows). After the vehicle detection phase, the vehicles are 
tracked until they reach the classification line. Then the vehicle dimensions are utilized to classify the vehicles 
into three classes (cars, bikes, trucks). After the classification phase, the vehicles in each class will be counted. 
The counting results can be used to estimate the traffic density at intersections, and adjust the timing of traffic 
light for the next light cycle. The system is applied on videos obtained by stationary cameras. The rest of 
paper is organized as follows: section 2 describes the vehicle detection and shadow removal algorithm. The 
vehicle classification method is presented in section 3. Experimental result is presented in section 4. Finally, 
conclusion is given in sections 5. 


2. VEHICLE DETECTION 


Moving object detection methods including human [6, 7], vehicles [8, 9], have been developed by sev- 
eral researchers. The most common method used in literature is the background subtraction method. In the past 
decade, numerous background subtraction algorithms have been proposed to extract and update the background 
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model. These algorithms can be a non-recursive or a recursive, the non recursive algorithm utilizes a buffer of 
video frames to get the background, while the recursive algorithm updates the background model recursively 
based on each input frame. Aljammal et al [10] proposed a non-recursive method called non-parametric model. 
In their method, they used the entire history of pixels to estimate the background by calculating the pixel den- 
sity function for each pixel. The pixel is considered as background 1f this function is greater than a predefined 
threshold. The drawbacks of this method are: it is time consuming and requires high memory storage .Kar- 
mann and von [11] developed a recursive technique for estimating the background model based on Kalman 
filter, they utilized the intensity and its temporal derivative model. The background is recursively updated by 
using three matrices: the background dynamics matrix, the measurement matrix and the Kalman gain matrix. 
However, this method is affected by the foreground pixels even if it works in slow rate adaptation. Kim et al 
[12] proposed a multimodal backgrounds method, it is called code book method. In this method, each pixel is 
summarized by a number of codewords stored in a coodbook; each codeword contains of a set of parameters. 
The input pixel is considered as a background pixel if its brightness falls within the brightness range of some 
codeword. In addition, the color distortion of that codeword is smaller than the detection threshold. However, 
this method cannot correctly detect the darkgrey and white moving objects, because it considers the darkgrey 
objects as a shadow and the white object as a sudden increase of illumination [13]. In this paper, MoGs method 
[14] is used to perform the background subtraction process. It is the most method used in literature due to its 
robustness against the environmental changes and its capability to handle multimodal background distributions. 
The idea of this method is as such: a number of Gaussian distributions (components) represent each pixel. The 
number of Gaussian components falls between three and five depending on the storage limitation and the pos- 
sibility of system realization in real time, three components are sufficient for our system. The component is 
considered as a matched component if the difference between the component mean and the pixel value is less 
than a predefined threshold, and then its parameters are updated as follows: the weight increases, the standard 
deviation decreases, and the mean moves to be close to the pixel value. If the component is none matched; 
the only parameter which is updated, is the weight, it decreases exponentially. If the pixel does not have any 
matched component; the component that has least weight is replaced by a new component with mean equals 
the pixel value, a large initial variance, and a small weight. After that the components are ranked according to 
a confidence metric (weight/standard deviation), and then a predefined threshold is applied to the components 
weights. The background model is the first components, whose weights are higher than the threshold. While 
the foreground pixels (moving objects pixels) are those that do not have any component in the background 
model. Figure 1 shows the result of applying MoGs method to two different traffic scenes. 





(e) (f) (g) (h) 


Figure 1. Vehicles detection in different scenes, (a) Scenel, frame1938, (b) Scenel, frame2015, 
(c) Scene2, frame1056, (d) Scene2, frame1733, (e) MoG result, frame 1938, (f) MoG result, frame 2015 
(g) MoG result, frame 1056, (h) MoG result, frame 1733 
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2.1. Shadow removal 
In this section, an algorithm that removes vehicle shadows will be developed to improve the perfor- 
mance of the system. The system performance will be improved from two respective views: It reduces the cases 
of occlusion problems between the vehicles. In addition, if the shadow region is not removed, it will be consid- 
ered as a part of vehicle dimensions, and the vehicle will be classified into the wrong class. Thus, the shadow 
removal algorithm leads to improve the classification process and increases the system performances. Many 
methods have been developed to detect and remove vehicle shadows [15-19], some methods utilize the color 
information to distinguish between vehicle pixels and shadow pixels because of the difference of the chromatic 
and luminance between the shadow and vehicle [20, 21]. However, these methods fail to identify shadow pixels 
from dark vehicle pixels. The other methods employ the texture information to identify the vehicle region [22]. 
Since the shadow has a little texture, it is simple to separate the vehicle region from the shadow region. In this 
paper, we proposed an algorithm to remove the shadow region based on the edges of detected regions. The 
proposed algorithm involves the following steps: 
— When the background subtraction method is applied, the moving objects (vehicles) are detected with their 
shadows (detected regions). Then the shadow removal algorithm is applied on these detected regions in 
the resulted image from background subtraction method and the gray scale source image. Figure 2 (a) 
and (b) shows the detected region in grey scale image and foreground image respectively. 


— The Canny edge detector is applied on these two images. The results of edge detection are shown in 
Figure2 (c) and (d) respectively. 

— XOR operation is applied on the two images resulted from the edge detection operation. Figure 2 (e) 
shows the image obtained from this operation. 


— Due to applying the edge detection on the detected region in the gray scale image, it may exist undesired 
background edges like edges of white marks, damaged marks, and trees or buildings shadows. Therefore, 
Canny edge detector is applied on the detected region in the background model (the image when there 
are no vehicles). 


— The edges of the background objects are subtracted from the edges of the image obtained from step 3, 
and what will remain are vehicle edges. Then we apply close operation on the resulted image to fill in 
spaces between the edges to preserve the vehicle details. 


The resulted image obtained from this step is shown in Figure 2 (P). 





Figure 2. Shadow removal 
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2.2. Vehicle tracking 

There are several methods to track moving objects through image sequences. These methods can be 
grouped into two categories: feature-based tracking methods, and model-based tracking methods. The feature- 
based methods are most widely used in literature because of its robustness [2]. In this work, after the detection 
of vehicles without shadows, a feature-based method is employed to track the moving vehicles through the 
image sequences in the detection zone (that is designated by two blue horizontal lines as shown in Figure 3). 
When the vehicle appears in the detection area, the centroid of vehicle is calculated, and then this feature is used 
to track the detected vehicle between the consecutive frames. First, an empty vector is initialized to maintain 
the position of vehicles centroid. If moving vehicles are detected in the current frame, their centroid position 
are recorded in this vector. In adjacent frames, the moving objects that are spatially closest are correlated. 
Therefore, the measurement of the distance between the consecutive frames is sufficient to track the moving 
objects. The euclidean distance (ED) is utilized to measure the distance between the position of vehicle centroid 
in the current frame and in the previous frame. For each moving vehicle in the current frame, a vehicle with 
the minimum distance is searched for the previous frame to change its record to the new position. When the 
vehicle exceeds the detection area, the record of this vehicle is removed from the vector. 





(c) (d) 


Figure 3. The results of the detection and classification process, (a) The result of the frame 438, 
(b) The result of the frame 350, (c) The result of the frame 1005, (d) The result of the frame 1572 


3. VEHICLE CLASSIFICATION 

After the detection of vehicles without shadows, the tracking process is implemented. When the 
vehicle reaches the classification line (that is designated by a red horizontal line as shown in Figure 3), the 
classification process is implemented based on vehicle dimensions. In this work, three parameters (aspect 
ratio (AR) = height/width, height, and width) of vehicles are utilized to classify them into three classes (cars, 
bikes, and trucks). The aspect ratio is calculated by using the dimensions of different types of vehicles; vehicle 
dimensions are taken from [23]. We took into our consideration the transformation from 3D to 2D. The aspect 
ratio of different vehicles types are as follows: 


— AR of cars is between [1.17-1.4]. 
— AR of tracks is between [1.3-1.9]. 
— AR of bikes is between [1.8-2.4]. 
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The classification process involves the following steps: 

— When the vehicle attains the classification line, its height and width are calculated. 

— If the vehicle width is greater than a specific threshold; then there is a horizontal occlusion (when the 
distance between the two adjacent vehicles is very small; they are detected as one object. This is called 
a horizontal occlusion). The detected region is considered as two adjacent vehicles, and it is separated 
into two regions by dividing the width by two. 

— If the vehicle height is greater than a specific threshold; then there is a vertical occlusion (when the 
distance between the two consecutive vehicles is very small; they are detected as one object. This is 
called a vertical occlusion). The detected region is considered as two consecutive vehicles, and it is 
separated into two regions by dividing the height by two. 

— The aspect ratio of each vehicle is calculated. If AR is between 1.17 and 1.3, then the vehicle is a car. If 
AR is between 1.41 and 1.79, then the vehicle is a truck. If AR is between 1.91 and 2.4, then the vehicle 
is a bike. If AR falls in the overlapping interval [1.3- 1.4], the vehicle height is used to distinguish if the 
vehicle is a car or a truck, if the height is higher than a predefined threshold, then the vehicle is a truck, 
else the vehicle is considered as a car. If AR falls in the overlapping interval [1.8-1.9], the vehicle width 
is used to distinguish if the vehicle is a bike or a truck, if the width is higher than a predefined threshold 
then the vehicle is a truck, else the vehicle is considered as a bike. 

— Three counters have been proposed to count the number of vehicles in each class. These counters are 
called C-car, C-bike, and C-truck. They are initialized to zero, and when the detected vehicle is classified 
into one class from the existing three classes, the counter of this class will be incremented by one. 


4. EXPERIMNTAL RESULT 

In this paper, a surveillance traffic system is proposed to detect and classify the different types of 
vehicles. In order to confirm that our proposed method is effective to perform this task, we utilized database 
that is taken from stationary traffic cameras of Casablanca city. This database contains of two different traffic 
videos; the first contains of 3580 frames with a resolution 240x320, the other contains of 2030 frames with the 
same resolution. Two different scenes of traffic videos are utilized. One video is taken in the area just after a 
traffic light (Scene 1) and the other is taken in an urban road (Scene 2). The results obtained during realized 
system operation contains of vehicle type and vehicle number. Vehicle number is displayed above the boundary 
box. Each vehicle is labelled by a black rectangular box until they reach the classification line, then the box 
color changes according to classification result. If the vehicle type is car, then the box color changes to blue. 
If the vehicle type is bike, then the box color changes to yellow. If the vehicle type is truck, then the box color 
changes to red. Figure 3 (a) shows the result of the 438-th frame in which 4 cars in the detection area, three 
of them crossed the classification line. They are classified as cars and have been labelled by blue rectangular 
box. Figure 3 (b) shows the result of the 350-th frame in which one truck has been counted and labelled by 
red rectangular box. Figure 3 (c) shows the result of the 1005-th frame in which one bike has been counted 
and labelled by yellow rectangular box and one car has been counted and labelled by blue rectangular box. 
Figure 3 (d) shows the result of the 1572-th frame from video sequences 2 in which two cars have been de- 
tected. The first one crossed the classification line, so it has been counted and labelled by blue rectangular 
box. According to these results, the different types of vehicles can be classified and counted correctly in this 
proposed system. Table.1 shows the results of vehicle counting for two different traffic scenes. The results 
manifest that the average accuracy of the two different scenes is 96.78%. 


Table 1. Result of vehicle counting 


Scene | Scene 2 
Vehicle Type Count Error Accuracy’ Vehicle Type Count Error Accuracy 
Car 230 6 97.45% Car 147 3 97.59% 
Bike 26 1 96.2% Bike 19 1 94.7% 
Truck 18 1 94.73% Truck 10 0 100% 
Average 96.13% Average 97.43% 


To evaluate the efficiency of the proposed system, a comparative study is established on different 
surveillance systems. There are difficulty to make a fair comparative study due to the utilization of different 
database. Therefore, we conducted a qualitative comparative study instead of quantitative study. Table 2 shows 
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the results of the comparative study. As noted in this table, the counting accuracy of the proposed algorithm on 
an average of 96.78% denotes that the proposed algorithm is more efficient than the other compared algorithms. 


Table 2. Result of the comparative study 


Comparative [24] [2] [3] [25] our method 

methods 

Vehicles types Cars only Cars and bikes Bus, minibus, Cars only Cars, bikes, and trucks 
car, and truck 

Vehicle detection Optical flow Background Background Frame differencing Background subtraction 

method subtraction subtraction 

Vehicle classification X Features Hierarchical X Features extraction 

method extraction Multi-SV Ms 

Vehicle counting 94.04% 96.9% Scene 1: 93% 96.04% Scene 1: 96.13% 

accuracy Scene 2: 88% Scene 2: 97.43% 


Scene 3: 90% 
Scene 4: 95% 


5. CONCLUSION 


A system for vehicles detection and classification has been introduced in this paper. The system 
consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle 
detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaus- 
sians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase 
and eliminate the undesired detected region. Then the vehicles are tracked until they reach the classification 
line. After that, the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, and 
trucks). After the classification phase, the vehicles in each class will be counted. The system has been applied 
on various traffic scenes under different weather and lighting conditions. The experimental results confirm 
that the proposed system has the ability in detecting and classifying the vehicles accurately and efficiently in 
real time. 
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